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WHAT IS CLAIMED IS: 

1 . A learning item sequencing system for optimizing a student' s learning 
speed, the system comprising: 
5 a computer including one or more memory portions; 

the memory portions comprising: 

a problem database containing a plurality of learning items, wherein a 
learning item is presented on each learning trial; 

a trial record database, for storing response data regarding the student's 
}=I0 response to each learning item; 

Jjj software for implementing a trial loop, wherein the learning trials are 

H presented to the student and response data are collected; and 

Hj software for implementing a sequencing algorithm, wherein the algorithm 

« sequences the learning items to be presented as a function of the response data collected 
iU 5 from prior learning trials. 

KJSl 

H 2. The sequencing system of claim 1 , wherein the response data collected for 

each learning trial includes the student's accuracy in answering the presented learning 
item and the student's response speed if the item was correctly answered. 

20 

3 . The sequencing system of claim 2, wherein each learning trial includes a 
priority score associated with the particular learning item. 

4. The sequencing system of claim 3, wherein the sequencing algorithm 

25 continuously updates the priority score of each learning item based on the response data 
collected on the immediately preceding learning trial. 
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5. The sequencing system of claim 4, wherein the number of trials since a 
particular learning item was last presented is a variable used by the sequencing algorithm 
in continuously updating the priority score of each learning item. 

5 6. The sequencing system of claim 4, wherein the relative importance of the 

student's speed and accuracy in sequencing of learning items is parameter. 

7. The sequencing system of claim 5, wherein the software for implementing a 
trial loop includes a problem selector, the problem selector selecting the learning item 
OO with the highest priority score for presentation to the student. 
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8. The sequencing system of claim 6, wherein when a subset of the plurality of 

; - 

jg learning items have the same high priority score, the problem selector selects the learning 
j 5 item for presentation at random from the subset of learning items with the same high 
H5 priority score. 

}«* 9. The sequencing system of claim 6, wherein the sequencing algorithm 

increases the priority score for each incorrectly answered learning item, wherein the 
probability of each incorrectly answered learning item being again selected by the 
20 problem selector increases, whereby the delay in learning item reappearance decreases. 

10. The sequencing system of claim 6, wherein the sequencing algorithm 
decreases the priority score for each correctly answered learning item, wherein the 
probability of that learning item being again selected by the problem selector decreases, 
25 whereby the delay in learning item reoccurrence increases. 
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1 1 . The sequencing system of claim 10, wherein the sequencing algorithm 
decreases the priority score for each correctly answered learning item as a function of the 
student's response time in answering the question, wherein the delay in a particular 
learning item's reappearance increases as the student's response time to the particular 
learning item decreases. 

12. The sequencing system of claim 5, wherein the sequencing algorithm 
prevents the same learning item from recurring for a predetermined number of trials. 

13. The sequencing system of claim 6, wherein each learning item may be 
assigned an initial priority score. 

14. The sequencing system of claim 13, wherein the sequencing algorithm does 
not modify the priority score of each item until after its first presentation. 

15. The sequencing system of claim 14, wherein the initial order of appearance 
of all, or some, of the learning items may determined in advance by assigning all, or 
some, of the learning items initial priority scores in ascending or descending order. 

16. The sequencing system of claim 11, wherein the software for implementing 
the trial loop includes a learning item retirement feature, wherein the learning items are 
retired from the problem database upon meeting a predetermined learning criterion. 

17. The sequencing system of claim 16, wherein the learning criterion for each 
particular learning item is a function of the student's accuracy in answering the learning 
item, and response speed for each correct answer, over a predetermined number of 
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repetitions of the learning item. 

18. The sequencing system of claim 17, wherein a session resumption feature 
stores an individual learner's data for learning items, in terms of speed, accuracy, and 

5 problem retirement status for continuing the learning during another session at a later 
time. 

19. The sequencing system of claim 17, wherein the session resumption feature 
allows learning to continue at another time whence the retirement count of retired 

fflO learning items is reduced, bringing them back for review and "re-retiring" if performance 
m still meets learning criteria. 

20. The sequencing system of claim 17, wherein the session resumption feature 
f : allows previously retired learning items brought back for review to rejoin the active 

!I5 problem set if performance criteria do not meet the previously established learning 
t f criteria. 

21 . A learning item sequencing system for optimizing a student's learning, 
comprising: 

20 a computer including a central processing unit, a visual display device, at least one 

input device, and one or more memory portions; 
the memory portions comprising: 

a problem database, wherein a plurality of learning items for presentation to 
the student is stored in the database, and further wherein each learning item has an 
25 associated priority score; 
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a trial record database, for storing a record containing response data 
regarding the student's response to each learning item, the response data collected for 
each learning item including the student's accuracy in answering each learning item and 
the student's response speed for each correctly answered learning item, the response data 
further including the number of trials since each particular learning item was last 
presented; 

software for implementing a trial loop, wherein the learning trials are 
presented to the student and the response data are collected, the software including a 
problem selector, wherein the problem selector selects the learning-item with the highest 
priority score for presentation to the student; and 

software for implementing a sequencing algorithm, wherein the algorithm 
continuously updates the priority scores for each learning item as a function of the 
response data collected from the prior learning trials, the sequencing algorithm increasing 
the priority score for each incorrectly answered learning-item and decreasing the priority 
score for each correctly answered learning item, wherein the probability of a particular 
learning item being again selected by the problem selector increases or decreases as a 
function of priority score. 

22. The sequencing system of claim 21, wherein the sequencing algorithm 
continuously updates the priority score of each learning item based on the response data 
collected for the immediately proceeding learning trial. 

23. The sequencing system of claim 21, wherein when a subset of the plurality 
of learning items have the same high priority score, the problem selector selects the 
learning item for presentation at random from the subset of learning items with the same 
high priority score. 
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24. The sequencing system of claim 2 1 , wherein the sequencing algorithm 
decreases the priority score for each correctly answered learning item as a function of the 
student's response time in answering the question, wherein the faster the student's 
response time the greater the decrease in priority score. 

25. The sequencing system of claim 21, wherein the sequencing algorithm 
prevents the same learning item from recurring for a predetermined number of trials. 

26. The sequencing system of claim 21 , wherein the priority score associated 
with each learning item may be assigned an initial value. 

27. The sequencing system of claim 26, wherein the sequencing algorithm does 
not modify the priority score of each learning item until after the first presentation each 
particular learning item. 

28. The sequencing system of claim 26, wherein the initial order of appearance 
of all, or some, of the learning items may determined in advance by assigning all, or 
some, of the learning items initial priority scores in ascending or descending order. 

29. The sequencing system of claim 2 1 , wherein the software for implementing 
the trial loop includes a learning item retirement feature, wherein the learning items are 
retired from the problem database upon meeting a predetermined learning criterion. 

30. The sequencing system of claim 21, wherein the learning criterion for each 
particular learning item is a function of the student's accuracy in answering that learning 
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item, and response speed for each correct answer, over a predetermined number of 
repetitions of the learning item, 

31. A method for learning item sequencing for optimizing a student's learning 
speed, the method comprising: 

supplying a computer including one or more memory portions; 
the memory portions having software which performs the steps of: 

building and maintaining a problem database containing a plurality of 
learning items; 

building and maintaining a trial record database, for storing response data 
regarding the student's response to each learning item; 

implementing a trial loop, wherein the learning items are presented to the 
student and response data are collected; and 

implementing a sequencing algorithm, wherein the algorithm sequences the 
learning items to be presented as a function of the response data collected from prior 
learning trials. 

32. A perceptual learning system for teaching students to recognize diagnostic 
structure from varying instances, the system comprising: 

a computer including one or more memory portions; 

the memory portions comprising: 

a problem database containing a plurality of learning items which are 
examples of different concepts, which, with respect to any one concept, includes a 
number of positive instances containing diagnostic structural elements and a number of 
negative instances which do not contain the diagnostic structural elements of the concept; 
and 
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software for implementing a trial loop, wherein the learning items are 
presented to the student in the form of short speeded classification trials. 

33. The perceptual learning system of claim 32, wherein the problem database 
5 includes learning trials in the form of structure discovery trials and structure mapping 
trials, the structure discovery trials requiring the student to recognize the diagnostic 
structure that determines a concept in a particular learning domain, distinguish positive 
and negative instances of the concept, and process the distinguishing information rapidly, 
and the structure mapping trials requiring the student to associate a structure in one 
QO representational format with the structure as it appears in one or more different 
fy representational formats. 

l Z 34. The perceptual learning system of claim 33, wherein the problem database 

5 a contains numerous instances exemplifying a given concept to be learned, and the set of 
{% instances systematically varies in attributes and dimensions other than the diagnostic 
«P structure determining the concept. 

;ass. 

35. The perceptual learning system of claim 33, wherein for each concept to be 
learned, the problem database contains numerous instances not belonging to that concept, 

20 for use as negative instances in classification trials. 

36. The perceptual learning system of claim 33, wherein for each concept to be 
learned, the negative instances systematically vary in their attributes and dimensions and 
share with positive instances of the concept various attributes and dimensions other than 

25 the diagnostic structure features determining inclusion of the concept. 
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37. The perceptual learning system of claim 32, wherein the learning trials 
include a pattern classification response or a pattern comparison response. 

38. The perceptual learning system of claim 32, wherein the student is given 
feedback regarding his performance after each learning trial. 

39. The perceptual learning system of claim 32, further including a trial record 
database, for storing response data regarding the student's response to each learning item 
and the concept of which it is an instance, the response data including the accuracy of the 
student's answer to each learning item, and the speed of the student's response to each 
correctly answered learning item. 

40. The perceptual learning system of claim 39, wherein the software for 
implementing the trial loop includes a retirement feature for concepts, wherein the 
individual concepts are retired from the problem database upon meeting a predetermined 
learning criterion for instances of that concept, wherein the learning criterion is a function 
of the student's accuracy in answering novel instances of that concept, and response 
speed for each correct answer, over a predetermined number of-presentations of instances 
of that concept. 

4 1 . The perceptual learning system of claim 32, wherein the software for 
implementing a trial loop includes a problem selector, the problem selector selecting each 
learning item at random from the problem database. 

42. The perceptual learning system of claim 41, wherein the problem selector, 
selects each learning item from the problem database according to the system of claim 21. 
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43. A perceptual learning system for teaching students to recognize diagnostic 
structure from varying instances, the system comprising: 

a computer including a central processing unit, a visual display device, at least one 
5 input device, and one or more memory portions; 
the memory portions comprising: 

a problem database containing a plurality of learning items, wherein each learning 
item is in the form of a structure discovery trial or a structure mapping trial, the structure 
discovery trials requiring the student to recognize the diagnostic structure that determines 
QO a concept in a particular learning domain, distinguish positive and negative instances of 
the concept, and process the distinguishing information rapidly, and the structure mapping 
trials requiring the student to associate a structure in one representational format with the 
|<g. structure as it appears in one or more different representational formats; 

software for implementing a trial loop, wherein the learning trials are 
I H 5 presented to the student in the form of short speeded classification trials, wherein the 
;F classification trials include a pattern classification response or a pattern comparison 
M response. 

44. The perceptual learning system of claim 43, wherein the student is given 
20 feedback regarding his performance after each learning trial. 

45. The perceptual learning system of claim 43, further including a trial record 
database, for storing response data regarding the student's response to each learning trial, 
the response data including the accuracy of the student's answer to each learning trial, 

25 and the speed of the student's response to each correctly answered learning trial. 
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46. The perceptual learning system of claim 43, wherein the software for 
implementing the trial loop includes a concept retirement feature, wherein concepts are 
retired from the problem database upon meeting a predetermined learning criterion, 
wherein the learning criterion is a function of the student's accuracy in answering the 
instances of that concept, and response speed for each correct answer, over a 
predetermined number of repetitions of each concept. 

47. The sequencing system of claim 46, wherein the contributions to concept 
retirement of learner speed, accuracy, and number of presentations meeting speed and 
accuracy criteria, are parameter adjustable. 

48. The perceptual learning system of claim 43, wherein the software for 
implementing a trial loop includes a problem selector, the problem selector selecting a 
learning item for each learning trial at random from the problem database. 

49. The perceptual learning system of claim 43, wherein the problem selector, 
selects a learning item for each learning trial from the problem database according to the 
system of claim 17. 

50. A perceptual learning method for teaching students to recognize diagnostic 
structure from varying instances, comprising the steps of: 

supplying a computer including one or more memory portions; 

the memory portions having software which performs the steps of: 

building and maintaining a problem database containing a plurality of 
learning items which are examples of different concepts, which, with respect to any one 
concept, includes a number of positive instances containing diagnostic structural elements 
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and a number of negative instances which do not contain the diagnostic structural 
elements of the concept; and 

implementing a trial loop, wherein the learning trials are presented to the 
student in the form of short speeded classification trials. 

51. An automated, time-based hinting method for promoting learning and 
preventing student frustration, the method comprising the steps of: 

supplying a computer including a central processing unit, a visual display device, 
at least one input device, and one or more memory portions; 

the memory portions having software which performs the steps of: 

running a learning program; 

building and maintaining a database of hints 

presenting a student with a learning trial; 

selecting a hint if the student fails to respond in a predetermined period of time or 
fails to answer the learning item for that trial correctly; and 
presenting the hint to the student. 

52. The hinting method of claim 5 1 , further including the step of providing 
feedback to the student after the student has responded to the hint. 

53. The hinting method of claim 5 1 , wherein the software selects hints from the 
group consisting of inverse operation hints, known problem hints, easy problem hints, and 
solved problem hints. 

54. The hinting method of claim 5 1 , wherein the software includes a hint 
selector that selects hints at random from the database of hints. 
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55. The hinting method of claim 5 1 , wherein the software includes a hint 
selector that selects hints according to an algorithm. 

56. The hinting method of claim 53 , wherein the software includes a hint 
selector that keeps track of previously used hints and hint categories to ensure that 
different hint types and instances are used about equally often. 

57. An automated, time-based system for providing hints in order promote 
learning and prevent student frustration, the system comprising: 

a computer including a central processing unit, a visual display device, at least one 
input device, and one or more memory portions; 
the memory portions comprising: 
software for running a learning program; 
a hint database containing a plurality of hints; and 

software for implementing a hinting procedure, wherein a student is presented with 
a learning trial and a hint is selected and presented to the student if the student fails to 
respond in a predetermined period of time or fails to answer the learning item for that trial 
correctly. 

58. The hinting system of claim 57, wherein the hint procedure provides 
feedback to the student after the student has responded to the hint. 

59. The hinting system of claim 57, wherein hints are selected from the group 
consisting of inverse operation hints, known problem hints, easy problem hints, and 
solved problem hints. 
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60. The hinting system of claim 57, wherein the software includes a hint 
selector that selects hints at random from the database of hints. 

6 1 . The hinting system of claim 57, wherein the hint procedure includes a hint 
selector that selects hints at random from the database of hints. 

62. The hinting system of claim 57, wherein the hint procedure includes a hint 
selector that keeps track of previously used hints to ensure that different hint types are 
used about equally often. 
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